AN ANALYSIS OF COST OVERRUNS ON DEFENSE ACQUISITION CONTRACTS 


/ 



N95- 20749 


An Analysis of Cost Overruns on Defense 
Acquisition Contracts 

by David S. Christensen 


Donald J. Yockey, the former Under Secre- 
tary of Defense for Acquisition, has called for 
more realism in the defense acquisition pro- 
cess. More specifically, he has called for real- 
istic cost estimates. The hope is that more re- 
alistic estimates will help surface problems 
in enough time to resolve them. 

Based on a review of over 500 contracts, the 
Office of the Under Secretary of Defense for 
Acquisition has observed that once a contract 
is 15 percent complete it is highly unlikely to 
recover from a cost overrun. Despite this im- 
portant observation, contractor and govern- 
ment personnel often claim that their pro- 
grams are different. 

This article examines the history of cost 
overruns reported on 64 completed defense 
contracts. Its purpose is to formally test the 
observation of the Under Secretary. Results 
confirm the observation at the 95 percent 
level of confidence, and were generally insen- 
sitive to the contract type (price, cost), the 
contract phase (development, production), 
the type of weapon system (air, ground, sea), 
and the armed forces service (Air Force, 
Army, Navy) that managed the contract. 
After a review of terminology, concepts and 
related research for those unfamiliar with 
the area, the methodology, results and man- 
agerial implications are described. 

Background 

Jacques Gansler reports that the average 
cost overrun on defense acquisition contracts 
is 40 percent. Cost data on defense contracts 
are regularly reported on cost management 
reports prepared by defense contractors. 
These reports include the Cost Performance 
Report (CPR) and the Cost/Schedule Status 
Report. Department of Defense Instruction 
5000.2 requires the CPR on all contracts 
judged significant enough for Cost/Schedule 


Control Systems Criteria (C/SCSC). Signifi- 
cant contracts are research, evaluation, test 
and development contracts with estimated 
costs of $60 million or more, or procurement 
contracts with estimated costs of $250 mil- 
lion or more. Thus, a 40 percent cost overrun 
on a procurement contract that barely quali- 
fies as significant is at least $100 million dol- 
lars! 

The cost/schedule control systems criteria 
are not a system. Instead, they are minimal 
standards for contractors’ internal manage- 
ment control systems. The purpose of the cri- 
teria is to foster reliable decision-making by 
contractor and government personnel. One of 
the requirements is that data reported by the 
contractor be summarized from the same sys- 
tems that the contractors use for internal 
management. These and other requirements 
help ensure that the data submitted to the 
government is useful for decision making. 

Another requirement of the criteria is a dis- 
ciplined budgeting system. A time-phased 
budget of all the authorized work on the con- 
tract, termed the “Performance Measure- 
ment Baseline,” is developed by the contrac- 
tor. The baseline is simply the summation of 
budgets assigned to elements of work on the 
contract. Because each element of work has a 
schedule, the budget for the work is said to be 
“time-phased.” 

The time-phased budgets assigned to work 
elements, termed the “Budgeted Cost of 
Work Scheduled” (BCWS), form the basis for 
earned value measurement and reporting. 
Earned value, also termed the “Budget Cost 
of Work Performed” (BCWP), is the same 
number as BCWS. The only difference is 
when they are recorded. BCWS is recorded 
when work is planned to be accomplished, 
and BCWP is recorded when work is actually 
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accomplished. If work is accomplished at a 
time different than it is planned to be accom- 
plished, then a schedule variance is identi- 
fied. In a disciplined budgeting system, all 
significant variances are investigated in a 
timely manner. 

A schedule variance often signals a cost vari- 
ance. A cost variance is simply the difference 
between the budgeted cost of work performed 
(BCWP) and the actual cost of work per- 
formed (ACWP). As with the schedule vari- 
ance, the criteria require the timely investi- 
gation and reporting of significant cost vari- 
ances. The intent is that through the timely 
analysis of variances, problems will be cor- 
rected before they become serious. 

Figures 1 and 2 illustrate the relationship 
between the three basic data elements just 
described. The performance measurement 
baseline is the cumulative expression of 
BCWS. Against this baseline, performance 
(BCWP) and actual cost (ACWP) are mea- 
sured. Figure 1 illustrates the typical condi- 
tion of defense contracts: over budget and be- 
hind schedule. 



Figure 1 . The Current Cost Overrun 
and Overrun at Completion 


A cost overrun is an adverse cost variance. 
Figure 1 illustrates two kinds of cost over- 
runs, termed the “current overrun” and the 
“overrun at completion.” The current over- 
run is the adverse cost variance to date. The 
overrun at completion is the difference be- 
tween the total budget for all the work on the 
contract, termed the “Budget At Completion” 
(BAC) and the estimated final cost of the con- 
tract, termed the “Estimate At Completion” 
(EAC). Note that the overrun at completion 
is an estimate until the contract is complet- 
ed. As shown in Figure 2, at the end of the 
contract, BCWP equals BCWS and the cur- 
rent overrun is the final overrun. 
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Figure 2. The Final Cost Overrun 


The estimate at completion is an important 
number and is very controversial, largely be- 
cause there is literally an infinite number of 
possible EAC formulas. The criteria do not 
prescribe a particular formula or set of for- 
mulas; the choice is the contractor’s. The 
only requirement is that the estimate be ra- 
tional. 

Because rational people can disagree, the 
government will usually evaluate the rea- 
sonableness of the contractor’s estimate by 
computing a range of EACs. Unfortunately, 


114 




an Analysis of cost overruns on defense acquisition contracts 


there is little guidance on what constitutes a 
reasonable range. As a result, the projected 
overrun at completion supported by the gov- 
ernment program office is usually higher 
than the contractor’s estimate. Because the 
government program office is necessarily an 
advocate of their program, their estimate 
may also be unrealistically optimistic. 

One way to assess the reasonableness of the 
estimated overrun at completion is to com- 
pare it to the overrun to date. If the overrun 
at completion is less than the overrun to 
date, then the contractor or program office is 
optimistically projecting a cost recovery. 
Such was the case in the A- 12 program. In 
April of 1990 the A-12 was in full-scale de- 
velopment and was 37 percent complete. The 
contractors’ reported overrun at completion 
was $354 million. The overrun to date was 
$459 million. Thus, the A-12 contractors 
were predicting a recovery of $105 million. 
Although this may seem optimistic, it is im- 
possible to know for sure because the A-12 
was canceled in January of 1991. 

Is such optimism justified? More specifically, 
is it unrealistically optimistic for the predict- 
ed overrun at completion to be less than the 
overrun to date? Based on a review of cost 
overrun data on completed contracts, the an- 
swer is that such optimism is unrealistic 
with 95 percent confidence. 

What Prior Research Says 
There has been some research into this issue. 
Wayne Abba and Gary L. Christie, senior 
analysts at the Office of the Under Secretary 
of Defense for Acquisition, have observed: 

Given a contract is more than 15 percent 
complete, the [final] overrun at completion 
will not be less than the overrun to date, 
and the [final] percent overrun at comple- 
tion will be greater than the percent over- 
run to date. 


This observation is based on a review of cost 
data on over 500 completed contracts. The 
analysts are quick to point out, however, that 
timely management attention to adverse cost 
variances can reverse them, especially early 
in the program. The problem has been a fail- 
ure to use performance measurement data 
proactively. 

The assertion of Abba and Christie is based 
on a casual review of over 500 completed con- 
tracts. The results of two empirical studies 
support the assertion. Both Kirk Payne and 
Scott Heise established that once a contract 
is 20 percent complete, the cumulative Cost 
Performance Index (CPI) does not change by 
more than 10 percent; in fact, in most cases it 
only worsens. (For example, in April 1990, 
the A-12 program was 37 percent complete 
and reported a CPI of 0.77. By September, 
the program was 47 percent complete and its 
CPI was 0.72.) 

As shown in Equation 1, the Cost Perfor- 
mance Index is a ratio of BCWP to ACWP. 

CPI = BCWP / ACWP 

A CPI that is less than 1 means that for ev- 
ery dollar spent, less than one dollar of work 
is accomplished. It follows that when the cu- 
mulative CPI is less than 1, the contract is 
experiencing a cost overrun, and because an 
unfavorable cumulative CPI only worsens, a 
contract is not likely to recover from a cost 
overrun. 

Therefore, if the predicted overrun at com- 
pletion is less than the overrun to date, the 
contractor’s estimated final cost of the con- 
tract (EAC) is unrealistically optimistic. 
This study further establishes these results 
by examining the cost overrun history on 64 
completed contracts extracted from the De- 
fense Acquisition Executive Summary 
(DAES) database. 
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Methodology 

The DAES database has received summary 
data on completed contracts since 1977. Pres- 
ently, data is summarized from Cost Perfor- 
mance Reports by government program of- 
fices and sent to the Office of the Under Sec- 
retary for Acquisition as quarterly DAES Re- 
ports. The database is a fairly detailed source 
of information on the cost performance of 
U.S. defense acquisition contracts. It is also 
reasonably accurate because most of the con- 
tracts in the database are C/SCSC compliant. 

For this study, a sample of 64 completed con- 
tracts was extracted from the database. Al- 
though the sample was purely judgmental, it 
is considered sufficiently rich to generalize to 
any C/SCSC-compliant defense contract. Ta- 
ble 1 shows cost overrun data by various 
categories considered relevant to this study. 

Based on the Under Secretary’s assertion 
and the results of prior research, four hypo- 
theses were tested (Table 2). For Hypothesis 
1, the average final cost overrun in dollars 


(FCO$) exceeds the average cost overrun to 
date (CO$). Hypothesis 2 is the same, except 
the overruns are expressed in percentages. If 
these hypotheses are correct with statistical 
significance, then recoveries from cost over- 
runs are improbable with a certain level of 
confidence. For this study, the hypotheses 
were tested at the 95 percent level of confi- 
dence. 

Based on the results of our prior research in- 
volving estimates at completion, it was ex- 
pected that the results of the testing may be 
sensitive to the contract completion point 
and other factors specific to the contracts in 
the sample. Therefore, the hypotheses were 
systematically tested at nine contract com- 
pletion points (10 to 90 percent at 10 percent 
increments) for various categories within the 
sample. The categories examined were the 
contract type (fixed price, cost), the contract 
phase (development, production), the generic 
type of weapon system, (air, ground, sea), 
and the armed forces service that managed 
the contract (Air Force, Army, Navy). 


Table 1 . Final Cost Overrun on 64 Completed Contracts 


Contract 

Category 

Number 

Overrun ($Millions) 

Overrun (Percent) 

Avg 

Min 

Max 

Avg 

Min 

Max 

All 

64 

36 

-3 

493 

18 

-3 

109 

Army 

28 

21 

-3 

46 

20 

-3 

46 

Air Force 

18 

49 

-2 

407 

19 

-1 

109 

Navy 

18 

47 

0 

493 

13 

0 

46 

Air 

43 

45 

-3 

492 

18 

-3 

109 

Ground 

13 

23 

7 

42 

21 

5 

45 

Sea 

8 

12 

0 

36 

12 

0 

38 

Development 

25 

38 

-2 

407 

21 

-1 

109 

Production 

39 

35 

-3 

493 

16 

-3 

46 

Cost 

23 

41 

-2 

493 

14 

-1 

46 

Price 

41 

34 

-3 

407 

20 

-3 

109 
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The remaining hypotheses are related to the 
results of the referenced CPI stability studies 
which established that the cumulative CPI 
tends to worsen from the 20 percent comple- 
tion point. Here the hypothesis was that the 
average cost overrun tended to increase. To 
test this hypothesis, the average cost overrun 
(CO) was regressed against percent complete 
(x)- 

CO = a + px 


Table 2. Hypotheses 


Hypothesis 

Interpretation 

HI: FCO$>CO$ 

Recoveries from 
cost overruns ($) 
are improbable 

H2: FCO% >CO% 

Recoveries from 
cost overruns (%) 
are improbable 

H3: P$>0 

Cost overruns ($) 
tend to increase 

H4: p%>0 

Cost overruns (%) 
tend to increase 


If the resulting slope coefficient (0) is positive 
with statistical significance, then the hy- 
pothesis is accepted, which means that cost 
overruns tend to increase. In Hypothesis 3, 
the average cost overrun was in dollars; in 
Hypothesis 4, the average cost overrun was a 
percent. As with Hypotheses 1 and 2, Hypo- 
theses 3 and 4 were tested on the entire sam- 
ple and on various categories of the sample. 

Equations 3 and 4 define the current cost 
overrun and final cost overrun in dollars. 
Equations 5 and 6 define the overruns as per- 
centages. 

Current Overrun (CO$) = Cum ACWP - Cum BCWP 
Final Overrun (FO$) = Final ACWP - B AC 
Current Overrun Percent = 100* (CO$/Cum PCWP) 
Final Overrun Percent = 100* (FO$/BAC) 


The cost overruns were averaged for each 
category of the sample by dividing the num- 
ber of contracts in that category into the total 
overrun for that category. The averaging was 
done at various stages of completion ranging 
from 10 to 100 percent complete, where per- 
cent complete was defined as follows: 

Percent Complete = 100* (Cum BCWP/BAC) 

Data earlier than the 10 percent completion 
point were not considered sufficiently reli- 
able. It can take as long as one year from con- 
tract award for the contractor to demonstrate 
C/SCSC compliance. Until then the data on 
the Cost Performance Report is suspect. 

Results 

As shown in the remaining tables, the hypo- 
theses were generally confirmed at the 95 
percent level of confidence. Table 3 shows the 
results of testing Hypotheses 1 and 2 on the 
entire sample of 64 contracts. Recoveries 
from cost overruns expressed in either dol- 
lars or as a percentage are improbable, espe- 
cially cost overruns experienced between the 
10 to 70 percent completion points. Between 
these points the difference between the final 
cost overrun and the overrun to date was sta- 
tistically significant at confidence levels well 
above 95 percent. After the 70 percent com- 
pletion, the current overrun percent is neces- 
sarily much closer to the final overrun per- 
cent because monthly expenditures typically 
decrease as the work nears completion. 

Hypotheses 1 and 2 were also generally con- 
firmed for the categories of the sample exam- 
ined. In short, recoveries from cost overruns 
on defense contracts are highly improbable, 
regardless of the contract’s type, the con- 
tract’s phase, the type of weapon system, or 
the armed forces service that managed the 
contract. 
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Table 4. Cost Overruns Tend to Increase 


Contract 

Cost Overrun ($ Millions) 

Cost Overrun (Percent) 

Categories 

Slope (|3) 

SE 

t 

Slope (B) 

SE 

t 

All 

0.325 

0.020 

16.13 

0.198 

0.009 

22.09 

Army 

0.186 

0.013 

14.27 

0.234 

0.016 

15.08 

Air Force 

0.407 

0.034 

12.11 

0.180 

0.005 

36.11 

Navy 

0.459 

0.021 

21.38 

0.159 

0.013 

12.20 

Air 

0.416 

0.022 

18.71 

0.210 

0.010 

20.30 

Ground 

0.168 

0.024 

7.06 

0.193 

0.018 

11.00 

Sea 

0.095 

0.008 

12.57 

0.139 

0.013 

10.37 

Development 

0.318 

0.024 

13.37 

0.232 

0.008 

29.12 

Production 

0.330 

0.019 

17.18 

0.176 

0.012 

15.08 

Cost 

0.393 

0.018 

21.40 

0.166 

0.015 

10.88 

Price 

0.287 

0.022 

12.93 

0.215 

0.006 

34.48 


SE = Standard error of slope coefficient with the intercept forced to zero; 
t = t statistic; ta = .05 df = 8 = 1.895 


Table 4 shows the results of testing Hypothe- 
ses 3 and 4, and confirms that cost overruns 
on defense contracts tend to increase. The 
slope coefficients were greater than zero with 
statistical significance for the entire sample, 
and for each category of the sample that was 
examined. 

Managerial Implications 
The results of this research show that recov- 
eries from cost overruns on defense contracts 
are highly improbable, and that cost over- 
runs tend to worsen as a defense contract 
proceeds to completion. This was found to be 
true regardless of the type or phase of the 
contract, the type of weapon system, or the 
armed forces service that managed the con- 
tract. The results are consistent with the re- 
sults of related research involving the stabil- 
ity of the Cost Performance Index, and con- 
firm the observations of senior analysts at 
the Office of the Under Secretary of Defense 
for Acquisition. 

These results have strong managerial impli- 
cations for the project manager: more realis- 


tic projections of the final costs are needed. 
When the projected overrun at completion is 
less than the overrun to date, the projected 
overrun at completion is too optimistic. For- 
mer Under Secretary of Defense for Acquisi- 
tion Donald J. Yockey commented in 1991 on 
this issue: 

We can’t afford to understate, sit on, or 
cover up problems in any program — at any 
time — at any level. They must be brought 
forward. This includes not just “show 
stoppers” but also “show slowers.” I can’t 
stress this strongly enough. 

Without more realistic estimates, senior 
management may be lulled into a false sense 
of security about their programs and fail to 
take appropriate action to correct problems. 

Wayne Abba and Gary Christie, senior ana- 
lysts at the Office of the Under Secretary of 
Defense for Acquisition, have commented 
that although recoveries from cost overruns 
are improbable, they are possible, especially 
if management pays proper attention to 
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them. With proper attention adverse vari- 
ances have been reversed. 

Proper attention requires a timely and disci- 
plined analysis of variances as they are iden- 
tified. It also requires a proper culture. A 
“shoot the messenger” culture was partly re- 
sponsible for the delayed reporting of adverse 
information on the A-12 program. According- 
ly, senior management should make every 
effort to cultivate a healthy attitude regard- 
ing variance reporting. Managers are neces- 
sarily advocates of their projects. But this 
does not mean suppressing or delaying the 
communication of adverse information about 
their projects to senior decision-makers. 

It is not known if recoveries from cost over- 
runs on non-defense projects are also improb- 
able. Perhaps additional research can explore 
this issue. Technical and political problems 
that contribute to cost overruns on defense 
projects may not be relevant to non-defense 
projects; however, the “shoot the messenger” 
culture involved in the A-12 program is cer- 
tainly a potential problem in non-defense in- 
dustries. 

A related “cultural” factor that contributed 
to the cancellation of the A-12 was the natu- 
ral optimism of senior management. In testi- 


mony before Congress, Navy Secretary of De- 
fense Garret characterized the senior manag- 
ers involved in the A-12 program as “can do” 
people who did not admit to failure lightly. 
Although optimism has its place, it can be 
dangerous when it blinds the manager to the 
truth. 

Finally, social scientists like Barry Staw 
have extensively documented many real- 
world examples of “escalation error.” In 
these examples, the decision-maker is ex- 
tremely reluctant to cancel an ongoing pro- 
ject or switch to an alternative, despite exces- 
sive overruns or other compelling evidence 
that the project has failed or that the alterna- 
tive is superior to the present course of ac- 
tion. In some cases, the manager chooses to 
escalate commitment to the project by in- 
creasing the spending on the project. Re- 
searchers have attributed such behavior to 
psychological factors such as a myopic “can 
do” attitude or a need to “save face.” 

More recently, Chandra, Bushman and Dick- 
haut have suggested that escalation error is 
caused by the manager’s desire to protect 
his/her reputation in the managerial labor 
market. Given the adverse economic conse- 
quences of cost overruns, additional research 
in this area is needed. 
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